Search AI Products and News

A recent joint study by Google, Carnegie Mellon University, and MultiOn explores the application of synthetic data in training large language models. According to Epoch AI, a research institution focused on AI development, currently available high-quality text training data totals around 300 trillion tokens. However, with the rapid advancement of large models like ChatGPT, the demand for training data is growing exponentially, projected to exhaust existing resources by 2026. Therefore, synthetic data is becoming increasingly crucial.

Recently, Tencent Cloud TI Platform officially launched the highly anticipated DeepSeek series models, including the full version V3 with 671B parameters and the R1 original model, as well as a series of models distilled from DeepSeek-R1, with parameter sizes ranging from 70B to 1.5B. This initiative provides developers with powerful AI tool support and further promotes the popularization and application of large model technology.

Google Research recently announced the innovative 'Titans' series model architecture, achieving a groundbreaking 2 million token context length through bionic design, with plans to open-source related technologies in the future. The core innovation of this architecture is the introduction of a Deep Neural Long-Term Memory Module, inspired by the human memory system. Titans cleverly combines the rapid response capability of short-term memory with the persistence characteristics of long-term memory, while utilizing an attention mechanism to handle immediate context, forming an efficient information processing system.

Recently, a study led by the Austrian Institute of Complexity Science (CSH) revealed that despite large language models (LLMs) excelling in various tasks, they show significant shortcomings when tackling advanced history questions. The research team tested three leading models, including OpenAI's GPT-4, Meta's Llama, and Google's Gemini, with disappointing results. Note: The image was generated by AI, and the image is licensed by Midjourney to evaluate these models in history.

In recent years, significant progress has been made in artificial intelligence technology, but challenges still exist between computational efficiency and multi-functionality. Many advanced multi-modal models, such as GPT-4, typically require substantial computational resources, limiting their use on high-end servers and making it difficult for intelligent technologies to be effectively utilized on edge devices like smartphones and tablets. Furthermore, real-time processing of tasks such as video analysis or speech-to-text still faces technical barriers, highlighting the need for efficient and flexible AI models to achieve seamless performance under limited hardware conditions.

According to reports from Intelligent Emergence, Jiang Xu, the former senior algorithm lead at OpenAI, has announced the establishment of a new company, Light Robotics, focused on the development of embodied smart companion robots. Jiang Xu is a key contributor to GPT-4 and left OpenAI in 2023, founding Light Robotics in August of the same year. Image credit: Image generated by AI, image licensed through Midjourney. Throughout his career at OpenAI, Jiang Xu participated in several important projects.

Recently, the AI research company Epoch AI released an interactive simulator designed to simulate the computational power required to train large language models. Through this simulator, researchers found that while a 2012 outdated graphics card (like the GTX580) could be used to train GPT-4, the cost would be ten times that of modern hardware today. Epoch AI's research indicates that the number of floating-point operations (FLOP) required to train GPT-4 is between 1e25 and 1e26.

Recently, research teams from several Chinese scientific institutions have launched the super large-scale multimodal dataset named Infinity-MM, and based on this dataset, they have trained an outstanding AI model called Aquila-VL-2B. This breakthrough injects new momentum into the development of multimodal AI. The Infinity-MM dataset is astonishing in size, containing four major types of data: 10 million image descriptions, 24.4 million general visual instruction data, 6 million selected high-quality instruction data, and 3 million generated by G